ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics

Physical simulators have been widely used in robot planning and control. Among them, differentiable simulators are particularly favored, as they can be incorporated into gradient-based optimization algorithms that are efficient in solving inverse problems such as optimal control and motion planning. Simulating deformable objects is, however, more challenging compared to rigid body dynamics. The underlying physical laws of deformable objects are more complex, and the resulting systems have orders of magnitude more degrees of freedom and therefore they are significantly more computationally expensive to simulate. Computing gradients with respect to physical design or controller parameters is typically even more computationally challenging. In this paper, we propose a real-time, differentiable hybrid Lagrangian-Eulerian physical simulator for deformable objects, ChainQueen, based on the Moving Least Squares Material Point Method (MLS-MPM). MLS-MPM can simulate deformable objects including contact and can be seamlessly incorporated into inference, control and co-design systems. We demonstrate that our simulator achieves high precision in both forward simulation and backward gradient computation. We have successfully employed it in a diverse set of control tasks for soft robots, including problems with nearly 3,000 decision variables.Comment: In submission to ICRA 2019. Supplemental Video: https://www.youtube.com/watch?v=4IWD4iGIsB4 Project Page: https://github.com/yuanming-hu/ChainQuee

Neural Fields with Hard Constraints of Arbitrary Differential Order

While deep learning techniques have become extremely popular for solving a broad range of optimization problems, methods to enforce hard constraints during optimization, particularly on deep neural networks, remain underdeveloped. Inspired by the rich literature on meshless interpolation and its extension to spectral collocation methods in scientific computing, we develop a series of approaches for enforcing hard constraints on neural fields, which we refer to as Constrained Neural Fields (CNF). The constraints can be specified as a linear operator applied to the neural field and its derivatives. We also design specific model representations and training strategies for problems where standard models may encounter difficulties, such as conditioning of the system, memory consumption, and capacity of the network when being constrained. Our approaches are demonstrated in a wide range of real-world applications. Additionally, we develop a framework that enables highly efficient model and constraint specification, which can be readily applied to any downstream task where hard constraints need to be explicitly satisfied during optimization.Comment: 37th Conference on Neural Information Processing Systems (NeurIPS 2023

Multi-robot grasp planning for sequential assembly operations

This paper addresses the problem of finding robot configurations to grasp assembly parts during a sequence of collaborative assembly operations. We formulate the search for such configurations as a constraint satisfaction problem (CSP).Collision constraints in an operation and transfer constraints between operations determine the sets of feasible robot configurations. We show that solving the connected constraint graph with off-the-shelf CSP algorithms can quickly become infeasible even fora few sequential assembly operations. We present an algorithm which, through the assumption of feasible regrasps, divides the CSP into independent smaller problems that can be solved exponentially faster. The algorithm then uses local search techniques to improve this solution by removing a gradually increasing number of regrasps from the plan. The algorithm enables the user to stop the planner anytime and use the current best plan if the cost of removing regrasps from the plan exceeds the cost of executing those regrasps. We present simulation experiments to compare our algorithm’s performance toa naive algorithm which directly solves the connected constraint graph. We also present a physical robot system which uses the output of our planner to grasp and bring parts together in assembly configurations

Co-Optimization and Co-Learning Methods for Automated Design of Rigid and Soft Robots

Nature demonstrates an incredible diversity, capability, and complexity of life, with organisms that can robustly run, jump, and swim. Compared with their biological brethren, robotic "life" lacks rich dexterity or economy of motion, and their plainly simple designs indicate room for improvement. Unfortunately, a major barrier to creating similarly adept robots is the design process itself. Each aspect of robot design, including the (physical) body (e.g. actuation, sensing, geometry, materials) and the (cyber) brain (e.g. control, proprioception) is typically not integrated in a single design workflow, and a lack of fast, accurate, useful simulators leads to expensive, spiraling, hardware intensive iteration. This thesis introduces methods to marry all aspects of robot design into combined algorithms for holistic cyberphysical design. Core to this solution is co-optimization and co-learning methods which can simultaneously reason about different design domains and achieve locally optimal performance. This thesis further discusses considerations in modeling (via differentiable simulation) and realizability (through automated and semi-automated fabrication worfklows). We describe how this entire suite of capabilities from modeling to automated fabrication can be conceptualized in a complete end-to-end "robot design stack," providing full CAD-CAM computational design capabilities. We demonstrate these capabilities on for rigid, compliant, and soft robot design tasks, including locomotion, manipulation, and tactile sensing, and discuss the frontiers of this burgeoning field of computational robot design.Ph.D

Multi-Objective Graph Heuristic Search for Terrestrial Robot Design

We present methods for co-designing rigid robots over control and morphology (including discrete topology) over multiple objectives. Previous work has addressed problems in single-objective robot co-design or multi-objective control. However, the joint multi-objective co-design problem is extremely important for generating capable, versatile, algorithmically designed robots. In this work, we present Multi-Objective Graph Heuristic Search, which extends a single-objective graph heuristic search from previous work to enable a highly efficient multi-objective search in a combinatorial design topology space. Core to this approach, we introduce a new universal, multi-objective heuristic function based on graph neural networks that is able to effectively leverage learned information between different task trade-offs. We demonstrate our approach on six combinations of seven terrestrial locomotion and design tasks, including one three-objective example. We compare the captured Pareto fronts across different methods and demonstrate that our multi-objective graph heuristic search quantitatively and qualitatively outperforms other techniques.Comment: IEEE International Conference on Robotics and Automation (ICRA 2021

Advanced soft robot modeling in ChainQueen

Abstract We present extensions to ChainQueen, an open source, fully differentiable material point method simulator for soft robotics. Previous work established ChainQueen as a powerful tool for inference, control, and co-design for soft robotics. We detail enhancements to ChainQueen, allowing for more efficient simulation and optimization and expressive co-optimization over material properties and geometric parameters. We package our simulator extensions in an easy-to-use, modular application programming interface (API) with predefined observation models, controllers, actuators, optimizers, and geometric processing tools, making it simple to prototype complex experiments in 50 lines or fewer. We demonstrate the power of our simulator extensions in over nine simulated experiments.</jats:p

Co-Learning of Task and Sensor Placement for Soft Robotics

Unlike rigid robots which operate with compact degrees of freedom, soft robots must reason about an infinite dimensional state space. Mapping this continuum state space presents significant challenges, especially when working with a finite set of discrete sensors. Reconstructing the robot's state from these sparse inputs is challenging, especially since sensor location has a profound downstream impact on the richness of learned models for robotic tasks. In this work, we present a novel representation for co-learning sensor placement and complex tasks. Specifically, we present a neural architecture which processes on-board sensor information to learn a salient and sparse selection of placements for optimal task performance. We evaluate our model and learning algorithm on six soft robot morphologies for various supervised learning tasks, including tactile sensing and proprioception. We also highlight applications to soft robot motion subspace visualization and control. Our method demonstrates superior performance in task learning to algorithmic and human baselines while also learning sensor placements and latent spaces that are semantically meaningful

Controlling a team of robots with a single input

We present a novel end-to-end solution for distributed multirobot coordination that translates multitouch gestures into low-level control inputs for teams of robots. Highlighting the need for a holistic solution to the problem of scalable human control of multirobot teams, we present a novel control algorithm with provable guarantees on the robots' motion that lends itself well to input from modern tablet and smartphone interfaces. Concretely, we develop an iOS application in which the user is presented with a team of robots and a bounding box (prism). The user carefully translates and scales the prism in a virtual environment; these prism coordinates are wirelessly transferred to our server and then received as input to distributed onboard robot controllers. We develop a novel distributed multirobot control policy which provides guarantees on convergence to a goal with distance bounded linearly in the number of robots, and avoids interrobot collisions. This approach allows the human user to solve the cognitive tasks such as path planning, while leaving precise motion to the robots. Our system was tested in simulation and experiments, demonstrating its utility and effectiveness.United States. Office of Naval Research. Multidisciplinary University Research Initiative. (ANTIDOTE N00014-09-1-1031)United States. Office of Naval Research. Multidisciplinary University Research Initiative. (SMARTS N00014-09-1051)Boeing Compan